To solve the problem of complex background and robustness in facial expression recognition, a novel method for facial expression recognition was proposed, which combined Active Shape Model (ASM) differential texture features and Local Directional Pattern (LDP) features in decision-making level by Dempster-Shafer (DS) evidence theory. ASM differential texture features could shield the differences between individuals effectively. Meanwhile it could try to retain expression information. LDP is a robust feature descriptor, which computes the edge response values in different directions and used these to encode the image texture. So LDP features have strong anti-noise capability and can capture the subtle changes caused by facial expression. With the consideration of different expression recognition rates for different features, different weight coefficients were selected to calculate probability assignment value during the process of DS evidence fusion. By conducting the experiments on JAFFE database and Cohn-Kanade database, the average recognition of facial expression can reach to 97.08% and is 1% higher than the method using single LDP feature. The experimental results show that the recognition rate and the robustness of facial expression are promoted.